Papers with human disagreement
Thesis Proposal: Toward a Human-Centered and Perspective-Aware Framework for Reproducible ML Evaluation and AI Alignment (2026.acl-srw)
Copied to clipboard
| Challenge: | Disagreement arises from subjective human opinion and can vary with one’s identity, beliefs, and social environment. |
| Approach: | They propose a human-centered framework for reproducible ML evaluation and AI alignment that takes disagreement into account when building human-centric AI systems. |
| Outcome: | The proposed framework is based on a human-centered and perspective-aware framework for reproducible ML evaluation and AI alignment. |
Seeing All Sides: Multi-Perspective In-Context Learning for Subjective NLP (2026.findings-eacl)
Copied to clipboard
| Challenge: | Modern language models excel at factual reasoning but struggle with value diversity, authors say . task-sensitive tasks such as hate speech expose this limitation . human disagreement captures the diversity of plausible human perspectives, authors argue . |
| Approach: | They evaluate four large language models with human disagreements on five datasets . they find multi-perspective in-context learning outperforms standard prompting . |
| Outcome: | The proposed approach outperforms standard prompting on English labels while disaggregated soft predictions better align with human judgments in Arabic and Italian datasets. |